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occupational choices for use in post high school plans, has three 
programs: (1) introduction and vocational orientation; (2) 

educational orientation and (3) post high school educational search. 
The subjects were: (1) black and white; (2) male and female; and (3) 

college and non-college bound. An experimental group consisting of 
students who had used the system and a control group who were not 
exposed to the system were randomly selected. Results indicate that 
the system experience leads to higher vocational maturity. Also, it 
was used equally by black and white students and by college and 
non-college bound students. Male students used the system more 
frequently than female students. Generally, the students were 
positive about their experience with the system, as were their 
parents. The counselors agreed on the potential contributions of the 
system but urged the use of other occupational materials. (RSM) 
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THE EDUCATIONAL AND CAREER EXPLORATION SYSTEM: 

FIELD TRIAL AND EVALUATION* 

Martin J. Bohn, Jr., Richard H. Lindeman, 
and Donald E. Super 

Teachers College, Columbia University 

In system development the strength of the logic, the completeness and 
accuracy of programs and routines, and the proven laboratory performance of 
equipment are usually not sufficient to justify the system. One must 
ultimately ask: "Does it work in the setting for which it was intended?" 

This paper presents the results of a field test of a computer-assisted 
counseling system, conducted in a suburban high school. The questions asked 
were: Does the system function adequately? Does it affect students* voca- 

tional development? What reactions does it elicit from students, parents, 
counselors, and teachers? 

1 

The System 

The Educational and Career Exploration System (ECES) has its theoreti- 
cal basis in vocational development theory. As an individual matures, he 
passes through life stages which are characterized by mastery of tasks usually 
expected of people in these stages. During the Growth and Exploration stages, 
the individual is engaged in learning about himself and trying new experiences 
to determine the kind of person he wishes to be (Super, 1957)- Some of the 
decisions made during this time have far-reaching implications for his educa- 
tional and occupational future. To make the best decisions for himself, the 
individual must know himself, must know something about the occupational world 
and its demands, and must know something about the relationship between his 
characteristics and those required by occupations. 

* Paper read at the American Educational Research Association Meeting, 
Minneapolis, March 6, 1970. 
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ECES was developed within this rationale, with an emphasis on exploration 
for the individual (Bohn & Super, 1969 ; Minor, Myers, & Super, 1969 )* ^ was 

decided that the system should deal primarily with the exploratory phase, he 
appropriate for varying socioeconomic and readiness levels and he under the 
control of the student, permitting the student and the system to work in an 
interactive mode. The system was designed to provide the student with infor- 
mation, to help him see the implications of his educational and occupational 
choices, and to supply data for use in post-high school plans, either in 
further education or in the world of work. Thus, students should come to the 
counselor ready to work at a higher level of decision making. Also, the 
system could serve as a comprehensive, up-to-date library of personally rele- 
vant occupational information. 

The system itself was comprised of the following components: a film image 
display unit, a numeric keyboard, a typewriter printer, and a reference manual. 
The first three components were connected to a computer at a remote site. The 
programs in the system were divided into three main sections: I. Introduction 

and Vocational Orientation; II. Educational Orientation; and III. Post-High 
School Educational Search. 



Design of the Study 

The design of the study called for the system to be installed in a high 
sehoo3. and to be made available to students After 'use by the students, 
attempts would be made to assess the effects of the system on the students’ 
vocational development and to assess the reactions of people in contact with 
the system (Thompson, Lindeman, Clack, & Bohn, 1970). The field trial was 
conducted in The Montclair, New Jersey, High School, a comprehensive high 
school serving grades 9 through 12 and having an ethnically and socioeconom- 
ically heterogeneous population of 2,300 students. The population from which 
the subjects were drawn included the four grades of the high school, both sexes 
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black and white races, and college and non-college bound students. A stratified 
random sample yielded an experimental and a control group. The experimental 
group consisted of students who had the opportunity to try the different 
phases and routines of the system. Control subjects were similar to the 
experimentals , but were not exposed to the system. The composition of these 
groups is shown in Table 1 . 



Table 1 

Initial Samples of Experimental and Control Students Classified by 
Grade, Sex, Race, and Post-high School Plans 

Grade Sex Race Post-high Sch. Plans 





9 


10 


11 


12 


Male 


Female 


White 


Black College 


Non-college 


Experi- 

mental 


4o 


4o 


4o 


4o 


80 


80 


102 


56 


104 


56 


Control 


4o 


4o 


4o 


40 


80 


80 


102 


53 


104 


56 



Note: In subsequent analyses these numbers varied because of incomplete data* 

Data obtained from these students included information about their school 
performance (e.g., grade point average), their aptitudes and interests, and 
their educational and occupational plans. The standardized testing was conducted 
before assignment was made to either the experimental or control groups. After 
the field trial period., appropriate measures were administered a second time, 
providing before and after comparison data. In addition, the experimental group 
responded to a questionnaire which solicited their reactions to the ECES exper- 
ience. The experimental subjects routinely completed a student reaction form 
after each session on the system. 

Results of the Study 

The system was available to the students from March through May, 1969. 
During that time 156 students used the system, with the help of a system 
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monitor and a clerk from the guidance department. The results presented here 
describe use of the system, users of the system, effects on student vocational 
development, and students ’ reactions to the system. Reactions and opinions 
from parents, school counselors, and teachers are also reported. 

How was the system used ? Table 2 shows that white students and black 
students used the system about equally often, as did college bound and non-college 
bound students. Male students used the system more than did the female 
students (7.2 sessions vs, 6.3 sessions). Female, black, non-college students 
used the system less than any other group (an average of 4.5 sessions) and that 
male, non-college 9th and 10th grade students used it most (9.4 sessions). 

Also, 12th grade students had a lower average number of sessions than did the 
other grades (5.7 sessions). This lower utilization by 12th graders was prob- 
ably because the final section of ECES, Post-High School Educational Search, 
which would have been most useful to high school seniors planning to go to 
college, was introduced late in the academic year. 

Table 2 

Mean Student Usage of ECES by Grade, Ethnic Background, Future, 

Education Plans, and Sex (N=156) 

Grade Ethnic Background Future Ed. Plans Sex 



9 


10 


11 


12 


White 


Black 


College 


Non-college 


Male 


Female 


N 42 


39 


4l 


34 


100 


56 


102 


54 


78 


78 


Mean 7.7 


6.7 


6.8 


5.7 


7.0 


6.3 


6.8 


6.4 


7.2 


6.3 



ECES usage and vocational development . Using data in the pre- and post-trial 
student questionnaires, several measures of vocational maturity were developed 
and refined. These were: 

1. Vocational and Educational Planning Maturity (m) 

2. Knowledge of Decision-Making Principles (DP) 
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3. Amount of Decision-Making Information (Dll) 

4. Amount of Decision-Making Information (DI2) 

5. Quality of Decision-Making Information (DI3) 

6. Acceptance of Responsibility for Decisions (RD) 

Covariance analyses were used to compare the experimental and control 
groups on the post-test data, using pre-trial data as covariates. The general 
hypothesis was that as a result of ECES experience, the experimental students 
would score higher than the control groups on the measures of vocational 
maturity. 

Scores on the Amount of Decision-Making Information (Dll), information 
the student believes he knows, were consistent with the hypothesis. Experimental 
students showed a greater increase on that scale than did the control students. 
The scores are shown in Table 3. No significant differences between experimental 
and control subjects were found on the other scales. 



Table 3 

Pre- and Post-Trial Means for Experimental and Control 

Subjects on Dll Scale 



Group 


N 


Pre-Trial Mean 


Post-Trial Mean 


SD 


F 


Experimental 


125 


36.19 


43.55 


9.03 


10.59* 


Control 


122 


37.^5 


39.64 


9^7 





* with-L and 214 d.f., significant at .005 level. The F test is based 
on a 2 ? analysis of variance with independent variables: grade, sex, 
race, future plans, and treatment. 

Other Measures. The experimental and control groups were also compared on 




measures of realism of vocational and educational self-concepts. Discrepancies 
on the Occupational Trait List scored for self, ideal self, best liked occupation, 
and least liked occupation were used to test the hypotheses that experimental 
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students would have more realistic self concepts, vocational and educational, 
than would control students - The differences between the two groups were negli- 
gible, producing no support for the hypothesis that three months use of ECES had 
any effect on these variables. 

Similar results were found on the Hotter Internalism-Externalism Scale, the 
Super Work Values Inventory, and the Holland Vocational Preference Inventory, 
although significant differences between certain subgroups were found on these 
scales. For example, on the Economic Returns value there was a significant 
interaction between treatments and post-high school plans. Among the experi- 
mental college-bound students ' scores on this value increased while those of 
non-college-bound students decreased; in the control group college-bound 
students' scores remained the same while the non-college -bound showed a decrease. 
Significant differences involving treatment effects were found on three Holland 
vocational preference scales (Realistic, Conventional, and Enterprising). These 
results had not been hypothesized and did not fit readily into any theoretical 
framework. In general, then, it was not established that ECES experience 
tended to modify individuals' scores on these standard measures. 

Student Reactions to ECES . At the completion of the field trial the 
experimental students were given a questionnaire to ascertain their reactions to 
and their opinions of ECES. Questions were asked about their experiences with 
ECES, their evaluations of the experience, their opinions about the usefulness 
of the system, and their suggestions about future developments of the system. 

The responses were first analyzed for the entire group, and then according to 
the major variables of grade, sex, race, and post-high school plans. 

Students were generally positive about their experience with ECES. A 
majority of them believed that the system had lived up to their expectations 
and that the system could benefit all kinds of students. They found ECES 
helpful in their occupational and educational planning; it helped make them 



aware of some occupational alternatives as well as important factors in 
occupational decisions* They agreed less about the helpfulness of ECES in 
understanding factors in their educational planning. 

The students expressed a preference for a guidance program which incor- 
porates both ECES and counselors. In the thinking of the students, ECES could 
not replace the counselor; however, there were activities which students believed 
could be better performed by ECES and other activities which could best be 
performed by the counselors. ECES was preferred for presenting facts about 
occupations, and information about occupational possibilities. The counselor 
was seen as superior in helping to integrate these facts. Also, the counselor 
was seen as superior when understanding of the student and his individual 
situation were needed. In a sense, these findings are consistent with the 
original plans and expectations for ECES. The system was designed to complement 
guidance programs and to serve as an aid to counselors, to help them work more 
effectively with their students. The students perceived this as a good way for 
ECES to be included in the programs as they know them. The students were 
obviously aware that the computer system cannot do all things that a counselor 
can do, although there are some functions that perhaps could best be taken 
care of by the ECES 0 

Differences in Attitudes . Among the subgroups of students there were 
some significant differences in the attitudes toward the system. The non- 
college-bound students, on the average, tended to report more helpfulness 
from the system than did the college bound. Non-college -bound reported that 
the system had more impact, that it was more helpful in gaining an -understanding 
of their strengths and weaknesses, finding definite paths of action, and dealing 
with problems. More than the college bound group, these students reported feeding 
that the system tried to choose a major for them. In a similar way, black 
students more than white students found ECES more directive, particularly in 



regard to choosing an occupation for them. The 9th and 10th graders generally 
found the system more helpful than did the 11th and 12th graders. 

From the students * reactions it would seem that ECES is seen as more 
helpful by those students who typically have less information on which to- 
make future plans, specifically the underclassmen, the blacks, and the non- 
college-bound students. It may be that ECES or a system like it is most 
appropriate for these students. 

Parent Attitudes toward ECES 

A questionnaire was mailed to all parents of the experimental students, 
asking them about discussions about ECES with their children, their opinions 
of the helpfulness of the system, and their activity in planning with their 
children. The parents responding, either by mail or by interview represented 
approximately 70 i of the families. The interviewed sample showed that the 
nonrespondents were different from the respondents mainly in that they discussed 
ECES less with their children, and they were less involved with the guidance 
programs or the counselors in the schools. The nonrespondent parents reported 
that their children had received more help from ECES than did the respondent 
parents . 

The general reaction of the parents to ECES was positive and enthusiastic. 
The parents felt that the system had value and could be of use in the vocational 
and educational planning of their children. Parents of white, college -bound, 
male students were perhaps least enthusiastic, feeling that their children 
had little new knowledge from ECUS. 

A comparison between parents in the upper levels (e.g., accountants, 
engineers, and physicians), and in the lower levels (bookkeepers, foremen, 
and mechanics) showed that those in the lower levels were more positive about 
the effects of ECES. They reported receiving more help in_s£ei^ig connections 
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between high school and future plans, between interests and vocations, seeing 
new occupational and educational possibilities, making good decisions. 

Counselors and ECES 

Counselors from the school became familiar with the system in the labor- 
atory, and in the school, before any students were enrolled in the experiment. 

To assess the effects ECES might have on the work of the counselors, and to 
find out their reactions to the effects which they perceived in their students, 
the counselors cooperated by completing a questionnaire before the field 
test, keeping a log during the test, and participating in a group interview 
on their opinions and reactions after the field test. 

The Pretest Questionnaire reflected the counselors ’ opinion that 
students often make their plans on the basis of limited information, without 
a good working knowledge of the demand for people in occupations they were 
considering, and little actual working experience in these occupations. The 
counselors believed that there is not enough time for them to work with their 
students and that they could work more effectively with the students if the 
latter had more and better information. The counselors were unanimous in 
their agreement that there should be more educational and career guidance. 

On the Counselor Interaction Logs there was little difference between 
interviews with experimental and control students. The major activity of the 
interviews was giving information, and most interviews moved in the direction 
of expanding the alternatives being considered by the students. The control 
groups received higher ratings on their use of the counselors 1 time and the 
students’ progress in thinking about the future. This suggests the need for 
further study of system effects on student-counselor relationships, especially 
over a longer time span. 

In the group interview with the counselors, they indicated having maintained 
a "hands off" attitude in regard to the system since this had been a field test. 



Because of this attitude , the counselors minimized their discussion of the 
system with students, although they did have some definite reactions and 
recounted specific incidents of student impact of the system. Their students 
had reported having specific in +<vl *ests awakened after being on the system. 

Some students had begun the program with explicit negative attitudes and in 
the process of the test reversed their attitudes. In the records of one 
counselor who had kept his student contacts, more than half of the students 
using the system made significant changes in their occupational plans, changes 
in the direction of more appropriate levels of education or wiser choices of 
occupational goals. 

The counselors reported a general increase in the use of other available 
occupational material available in their departmental library, among both 
experimental and control students. Counselors were able to see the potential 
contributions such a system could make to a school guidance program. It is 
perhaps with counselors that most work must be done to effectively introduce 
>. computer -assisted guidance system into the schools. 
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